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IhfTEGRATING BEST PRACTICE AND PERFORMANCE INDICATORS 
TO BENCHMARK THE PERFORMANCE OF A SCHOOL SYSTEM 



Introduction 

This paper synthesises the literature on the role of benchmarking with a 
particular focus on Its use In the public sector. Benchmarking Is discussed In 
the context of quality systems, of wtilch it Is an Important component. The 
basic types of benchmarking are discussed along with research on Its 
application In the public sector. The purposes of performance indicators are 
addressed along with the types of lnformatk)n that they can provide. Finally, 
benchmarking and performance Indicators are brought together In the 
description of the benchmarking framework that Is an Integral part of quality 
assurance in the New South Wales (NSW) Department of School Education. 
Benchmarking provides the conceptual framework to Integrate best practice 
and performance Indicators Into a vital component of this quality assurance 
system. 



Best Practices in Quality Managenrvant 

Overview of Qu&lity Systems 

Quality systems can be viewed as having passed through four generations 
of development. The first generation employed quality management strategies 
based on concepts of quality control that relied primarily on Inspection of final 
products and had almost no application outside the process manufacturing 
sector (circa 1950s and 1960s). 

The second generation of development employed quality management 
strategies which today would be viewed as quality assurance In the narrow 
definition of that term. They were essentially based on the accreditation or 
certification of manufacturing processes against specified process and product 
standards. Accreditation of organisations and processes against national and 
international standards (eg. AS3901, ISO9000) are examples of such strategies 
in use today. 

The third generation of quality management strategies relates to a range of 
disparate approaches which are often referred to collectively as total quality 
management strategies These fccus on both process and product standards 
and the 'building in' of quality through managing continuous improvement in 
processes They place significant emphasis on customer satisfaction, waste 
reduction, routinisation of practices and teamwork (McLagan, 1991). A number 
of recent reports of attempts to apply such quality management practices to 
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schooling have appeared in the literature (Dudden. 1993; Hough. 1993; Lezotte 
1992). There are a range of issue that have not been resolved in terms of the 
appropriateness of applying such quality management strategies to the 
development of schools (Capper and Jamieson. 1993; Cuttance. 1993a; see 
also the special edition of Educational Leadership of September 1993). 

Current research suggests the need for a new fourth generation of quality 
management strategies. This generation should recognise that different quality 
management strategies are required at different phases of the performance 
development cycle in organisations. The message here is that, contrary to the 
received wisdom of third generation practices, there are few universal quality 
management strategies that are applicable across all stages of an 
organisation's development. 

An international study of 945 quality management strategies in 580 
commercial and industrial organisations found that only three quality 
management strategies had universal application across organisations at 
different points in their performance development cycle (American Quality 
Foundation. 1992). These were: 

• a strategic focus on process improvement; 

• strategic planning; and 

• supplier certification programs. 

The most important finding from this research was that different quality 
management strategies were effective in different phases of the organisational 
performance development cycle. In the early part of the cycle the strategies 
which were most effective in improving performance were: 

• building teams; 

• empowering staff to solve problems; 

• general and specific training; and 

• a strong emphasis on inspection of the product. 

Organisations in the middle of their performance development cycle 

benefited most from: 

• the use of teams, a continued emphasis on training; 

• a focus on process improvement through its simplification; 

• the implementation of vendor-certification programs; 

• the use of quality assurance systems to enforce compliance with 
process and service standards; 

• tight control over strategic planning; and 

• the monitoring of progress against targets. 

Organisations approaching the top of the performance development cycle 

gained most advantage from: 

• empowering employees to interact directly with their customers; 

• undertaking benchmarking studies; 

• implementing piocess simplification; and 
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• making innovation and creativity the focus of quality assurance 
strategies. 

The vital message from this research is that organisations need to change 
their quality management strategies as they progress through their 
performance development cycle. The strategies which are effective for 
improving performance at one stage of the cycle are not necessarily effective at 
other stages of the cycle. These findings, of course, run counter to the 
received wisdom of current quality management practices in the business 
sector. In particular, the finding that there are indeed few universal quality 
management strategies which are effective at different stages of organisational 
performance development runs counter to the main emphasis of such 
approaches as total quality management. 

It is worth quoting the conclusions on benchmarking in the American Quality 
Foundation Report in full: 

[The findings show that] organisations ere adopting benchmarking practices with 
irKreasing frequertcy 

However, the twst practice analysis shows that benefits from benchmarking are 
restrk:ted to only the higher-performing organisations. This group shows 
panicuiarty high positive impacts from benchmarking their delivery and distribution 
systems They also show immediate benefits from benchmarking their marketing 
and customer service systems. 

In the k}wer- and medium-performing groups, there is no compelling positive 
impact from any of the berKhmarking practices. The k)wer group actually shows a 
negative impact from benchmarking marketing and sales systems. ^ 

We believe that there are two reasons why k)wer-pefforming organisations do nor 
benefit greatly from bertchmarking practices. Rrst they are likety to be toolatrg at 
inappropriate role models. The common practice in berKhmarking is to examine 
the t>est in the workT or workl-ciass organisations. Yet the IQS data have shown 
again and again that the practices that distinguish Ngher-performing organisatk>ns 
are almost always ineffective when adopted by k>wer-performing organlsatk)ns. 
Lower performers probably wouM find organisatons that are on the threshoM of 
medium performance, rather than worid^lass organisatkxis, to be more helpful 
models Second, the lower-performing organisations need to focus their resources 
on their core infrastructure and not diffuse their focus with the sophistk:ated 
practices they would see in the best of the best. (p38) 



Benchmarking 

Background 

Benchmarking is one ol a range ol strategies that have been developed 
over the last decade or so to assist organisations to assess and develop their 
performance Harrington (1991) discusses benchmarking as one of a set of 
strategies available for business process improvement. Some writers treat 
strategies such as reengmeering, benchmarking and Total Quality 
Management (TOM) as alternatives and others seek to integrate them. For 
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example, the major text on benchmarking (Camp, 1989) makes no reference to 
TQM, while other writers such as Wald (1993) ir discussing Spendolini's (1992) 
7776 Benchmarking Book describes benchmarking as the cornerstone of TQM. 
Others claim benchmarking is the parent framework and provide a reference to 
strategies such as TQM, Quality Systems, Kaizen, etc as part of the overall 
benchmarking framework (McGonagle and Fleming, 1993). This lack of 
integration of the various strategies reflects the heritage of the multitude of 
approaches to organisational improvement that have been generated largely 
from the experience of particular Indivkjuals and firms. 

Benchmarking as it is known in the business literature of the 1990s arises 
from the work of Keams and his colleagues at Xerox in 1979, notwithstanding 
claims that the Western workj has finally discovered the tool that Japan used 
so successfully to ck)se the quality gap in the 19608 and 70s' (Fitz-enz, 1992). 
Keams was a prime mover in having benchmarking included in the criteria for 
the Baldridge Award, the major quality award in the USA 



Definition of Benchmarking 

The concept of benchmarking has entered the common language, as is 
exemplified by the Australian Council of State School Organisations (ACSSO) 
newsletter article Illusory Savings from Benchmarking in February 1994. This 
article commented on the agreement at the 1993 Premier's Conference to 
establish a joint Commonwealth/State Working Party to review Commonwealth 
and State sen/ice provision by the collection and publicatkjh of data to allow 
benchmarking comparisons of efficiency. The recent announcement of the 
publication of Report Cards for Hospitals (Sydney Morning Herald, 22 March 
1994) reflects this strategy of provkJing benchmark information to guide public 
choice in particular spheres of the provision of public services. 

The ACSSO article describes benchmarking in the folk>wing terms: 

Benchmarking for government sennces such as health and edusatran involves a 
comparison of ifKjicators of service performance between different governments. 
Performance indicators measure such aspects as the cost of proviskx), the level of 
service and its quality. Data on these irxiicators are coRected and published to 
allow comparisons between governments. Benchmaricsofperfor.nartce may then 
be established aooordir>g to prendetermined goals such as the towest-oost provider, 
the most efficient, the best quality or the most equitable provision. Performance 
indicators and benchmarfu are not neutral technical exercises. What is to be 
measured depends on the objectives sought in provkjing a service. Different policy 
objectives require different performance indcators to assess vyfwther the 
objectives are being met {ACSSO News, Vol 3, pi , February 1 994) 

This description provides a reasonable guide to what is meant by 
benchmarking, but it has a strong consumer and public sector flavour when 
compared to the working definitions used in business. Kearns (1989) 
described benchmarking as the way an organisation continually compares itself 
with industry leaders to help it learn about hcsv it can achieve high 
performance The International Benchmarking Clearinghouse describes 



ERIC 



5 



benchmarking as a process in which companies target key improvement areas 
within their firms, identify and study best practices^ by others, and implement 
new processes and systems to enhance their own productivity and quality 
(Mittelstaedt, 1992). 

Features of Benchmarking 

Three quarters of Fortune 500 companies in the USA engage in 
benchmarking (Mittelstaedt, 1992). They focus their benchmarking strategies 
on both processes and outputs. Tucker, Zivan and Camp (1987) suggest that 
managers tend to focus at first on comparing costs for parttoular outputs out as 
they become more familiar with benchmarking they discover that understanding 
practices, processes and methods is more important because they define the 
changes necessary to reach the benchmark outcomes. 

Benchmarking can have either internal or external referents, and external 
referents can be either national or international. For example, the Australian 
Federal Government Best Practices Program promotes international 
benchmarking of Australian fimis. A range of Australian firms, such as, 
Qantas, Commonwealth Industrial Gases. Australian Newsprint Mills Ltd and 
Henderson Automotive (SA) have been supported by the program o 
benchmark their processes against best-in-the-worid standards.^ 

There are three types of comparative referent groups. Internal referents are 
to other units within the organisation. External referent grpups include both 
competitors and best-in-class in other industries. 

There are essentially four types of benchmarking: 

• competitive benchmarking 

• internal benchmarking 

• functional benchmarking 

• generic benchmarking (Camp, 1989). 

Compatitlv* Benchmarking 

There are a number of traps to be avoided with competitive benchmarking 
Integrity of the data used for comparisons is paramount, but it is often difficult to 
verify this in a true competitor situation. For this reason, a significant amount of 
competitor benchmarking is based on relatively high level outcomes. The data 
for comparisons can be gleaned from a range of official and public sources, 



^ The identification of besr pracrtce is often taken to be unproblematic It is. of course, nothing 
of the sort The science of determining the most effective practices in an organisation arxj 
assessing which are more effective than others raises questions of interdependence m 
systems and of research methodotogy, among others It is much more difficult in a 
non-experimental situation to ascerlain the relative effects of pons of a system than is 
commonly supposed: systems theorists would argue that it is an ontotogical impossibility 

2 See the article in Century 21, No 5, p90-94, August 1992 
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such as the annual accounts of companies. In principle, this should be less of 
a problem with comparisons between public sector organisations as they are 
often not direct competitors. However, they may exhibit many of the 
sensitivities of competitor organisations. In the long run most public sector 
organisations are in competition with other parts of the public sector for budget 
share. Further, many public sector organisations have direct competitors in the 
private sector, non-government schools, for example. 

Another issue is that an organisation needs to benchmark against one that 
is better than Itself. The first problem is determining which competitors are 
performing better. This may be easy for low performance organisations, but 
difficult for high performance organisations. Not only do high perforrrance 
organisations tend to be more interested in productivity and quality 
improvement strategies such as benchmarking, they have fewer competitors 
worth benchmarking against. Evidence on the quality improvement strategies 
with highest marginal productivity suggests organisations with tow perfomiance 
would gain most from benchmarking themselves against organisations which 
are ahead of them in performance, but not so far ahead as to make 
comparisons incommensurate (Cuttance, 1994a). Best practice organisations 
have few competitors against whom it is worthwhile to benchmark — they 
already surpass the output performance and process standards of most of their 
competitor organisations. 

High performance organisations that practice tjenchmarking have come to 
the realisation that they will be the recipients of a surfeit X)f requests to be 
partners in benchmarking exercises. Many of these requests will come from 
organisations that have little to offer them in terms of process improvement or 
increased productivity. 

A final problem for competitive benchmarking to overcome relates to this 
last point. Finding out how your competitors at the same level or a marginally 
higher level of performance achieve their outcomes is not likely to do any more 
than allow you to tread water. It is certainly unlikely to provkJe the information 
to drive a substantial program of improvement the performance of your own 
organisation. The search for benchmarking partners is therefore destined to be 
somewhat one sided at the overall organisational level if restricted to 
competitors. This is one of the main reasons why functional and generic 
approaches to benchmarking have emerged. 

Internal Benchmarking 

As indicated earlier, internal benchmarking tends to be the first form of 
benchmarking that organisations engage in (McGonagle and Fleming. 1993). 
The problems are fewer but the payoffs may also be relatively low in most 
cases. 

In large organisations similar functions may be undertaken across a range 
of different sites or in various parts of the organisation Large conglomerates 



may have several pr(x;essing plants which carry out somewhat similar work. In 
the public sector, government school and hospital systems are examples of 
organisations with replicated and relatively autonomous operations across 
many sites. In such cases the lexicon of comparisons is likely to be better 
understood because of a common and shared understanding about the 
measurement of processes and outcomes. 

Although it is often assumed that it is easier to share 'confidential' 
information among the various operating units of an organisation, sensitive 
information often brings with it anxieties and accountability issues. 
Comparisons within an organisation are certainly likely to be less expensive 
and more easily arranged than between independent organisatkxis. However, 
this advantage should not be taken for granted — it can often be easier to 
make arrangements with an independent organisatton than with an internal 
competitor within ones own organisation. In the finish, we all comp)ete for 
status, our share of the resources, staff, etc — and the ethos of competition 
c'in sometimes be more strident within than between organisations. 

In the public sector internal benchmarking will have its greatest applicability 
in organisations that have a large number of autonomous and replicated 
operations and where there is significant variation in processes and 
performance among 0F>erating units. In Australia, school, police and hospital 
systems are likely candidates, as are the various operating sites for 
organisations such as the post office and transportation systems. The 
devolution of authority in school and hospital systems and the corporatisation of 
many of the larger public service organisations Is designed to improve 
performance through more efficient decision making and, in the case of 
corporatisation, through the introduction of internal competition (Osborne and 
Gaebler, 1993). 

A challenge for intemal benchmarking exercises is to ensure that they result 
in something more than a 'work harder' conclusion — one encompassing a 
smarter way of doing things because new connections have been made that 
lead to genuine innovatk>n. The strategy for benchmarking processes and 
outcomes described later for the NSW government school system incorporate 
the idea of best practice from throughout the school system into an intemal 
benchmarking strategy. Rather than directly compare itself with a another 
school each school can compare itself to best practice on a number of 
dimensions across all schools in the system. This provides a way of 
challenging the performance of individual schools, through a structured 
development and quality assurance review process 

Functk>nal Benchmarking 

Functional benchmarking is externally focussed but not on competitors It is 
the approach to benchmarking that is driven more by a spirit of innovation than 
of comparison The main strategy is to benchmark against organisations that 
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are leaders in their felid. The emphasis is on studying the reasons for the high 
performance of the organisations, through an analysis of their internal 
processes and strategic structures. This may result in the establishment of a 
lighthouse organisation through the adoption of practices that have not 
previously been thought of as being applicable in the particular industry. The 
adoption of bar coding in a range of Industries after Its development in the 
grocery industry is an example often cited. There were many examples of the 
cross-pollination of Ideas tor the use of electronic equipment in the early days 
of what has become known as computerisation. 

In this type of benchmarking the transfer of knowledge is often more at the 
level of concepts and strategies than at the level of specific processes. Many 
quality management strategies, such as a focus on the customer, can be 
thought of as having taken hold through the generatton of knowledge from a 
process similar to functkxial benchmarking. The strategy of multl-skilling was 
the spearhead of microeconomic reform in Australia during the 1980s. 
Multi-skilling originated in the metals industry and was transferred across to 
other process industries before being taken up in an adapted form by servk:e 
industries. Although not discovered through a benchmarking process the 
transfer of knowledge across industries of this strategy parallels the transfer of 
knowledge in functional benchmarking processes. 

G«n*iic Benchmarking 

This type of benchmarking applies to processes that are qomparable across 
organisations which may be in different industries. For example, the payroll 
function can be thought of as a generic process which is independent of 
industry. The payroll process in a hospital in many respects resembles that of 
a coal mine, or a car plant. Many administrative and management functkKis 
are generic in this sense. It is still important, however, to take context into 
account. The concept of niche markets for managers with 'specialist' 
experience is based on this fact. It is mainly contextual knowledge, not 
processes or technk^ues, that differentiates between a marketing manager of a 
retail store and a Leagues Club. 

The contextual environment is part of the equation in understanding 
differences in the efficiency of operatkxis across organisatk)ns. For example, a 
few years ago I investigated the relative performance of the personnel/payroll 
function in a car plant, a public service agency and a bank. These 
organisations exist to carry out very different tasks. The most efficient 
personnel/payroll section was in the car plant whteh had one member of staff 
per 450 employees The bank had a ratio of 1:350 and the public sector 
agency had a ratio of 1 :200. 

There were very significant differences in context. The car plant paid almost 
all employees weekly and had a modest employee turnover rate. The banK 
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and the public service agency paid a high proportion of their employees 
fortnightly and had very low employee turnover rates. 

The employee statistics used in the above ratios were based on effective full 
time equivalents. This had little effect on the car plant ratio, but had a 
significant effect on the public sector agency's ratio, as it had a proportion of its 
employees working part time. The low turnover rate should have favoured the 
efficiency ratio of the public sector agency and the bank, but the high number of 
part time employees appeared to outweigh any comparative advantage from 
this source. Other contextual factors that were considered at the time to 
explain the significant differences In effk:iency between the organisatk)ns were 
the complexity of the awards in the public sector agency and the higher level of 
staff training and devek)pment undertaken in that organisatk>n. 

This example shows why benchmarking does not always provice a clear 
answer. The main conclusion derived from this exercise was that the 
complexity of the award structure in the public sector agency was probably a 
primary factor for the additional one million dollars a year it cost to run the 
personnel/payroll section compared to the relative costs of an operatk)n based 
on the efficiency ratio for the car plant. This additional cost amounted to about 
$50 per employee per year.3 

The Benchmarking Process 

Benchmarking involves measuring an organisation's performance against 
the best-in-class organisatk>ns. To be effective the process must provkje an 
understanding of how the best-in-class organisations achieve their superior 
performance. This knowledge is then used as the basis for a strategic 
approach to the implementation of organisational developments that will 
achieve similar levels of performance in the organisation undertaking the 
benchmarking. Camp (1989) provides details of each stage of the 
benchmarking process. 

Benchmarking is directly linked to the missk>n as specified through the goals 
and objectives of an organisation. It is utilised in setting the level of 
per!ormance of the outcomes to be achieved and in the determination of the 
process improvements that are required to achieve these outcomes (Camp. 
1989; Pryor. 1989). Thus, the organisation's goals are themselves derived 
from benchmarks based on best practice. 



3 Compared to the $10,000* annual non-salary costs of entptoying staff in most organisattons 
this could be viewed as a minimal difference between the three organisations compared 
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The benchmarking process is set out in five stages by Camp (1989) as 
follows: 

[1] Planning 

• Identify what is to be benchmarked and the data required. 

• Identify comparative companies, or units/processes within the 
organisation. 

• Determine the da a collection method and collect the data. 
[2] Analysis 

• Determine the *gap' between current performance and best 
practice performance. 

• Project future performance levels. 

[3] Integration 

• Communicate benchmark findings and gain their acceptance. 

• Establish functional goals. 
(4J Action 

• Devek>p action plans. 

• Implement specific actions and monitor progress. 

• Recalibrate benchmarks by returning to [1 ]. 

[5] Maturity 

• Leadership action attained. 

• Practices fully integrated into processes. 

Benchmarking in the PubWc Sector 

Lamp (1989) provides five key reasons for benchmarking in the private 
sector: 

• meeting customer needs; 

• establishing high performance goals and objectives: 

• developing true measures of the organisation's performance; 

• improving competitiveness; and 

• achieving industry best practices. 

As a set of reasons for applying benchmarking these coukJ apply equally to 
many organisatwns In the public sector. Public sector organisatkxis may not 
be primarily in competition with others for customers or profits, but may be in a 
competitive situatk>n for budget share and resources. Compared to traditional 
management perspectives, benchmarking provides a more significant focus on 
the external environment — customers, competitiveness in the market and the 
best practices of other organisations. 

There has been an increasing focus on performance in the public sector 
over the last decade or so. This was the major theme of Osborne and 
Gaebler s (1993) important volume on Reinventing Government The 1993 



11 



Premier's Conference agreed to review service delivery across states using a 
benchmarking approach (Sydney Morning Herald, 5 July 1993). 

There are a number of reasons why public sector organisations have taken 
up strategies such as benchmarking to improve performance: 

• the boundaries between the public and private sectors are now less 
definitive 

• some public sector organisations are operating in a competitive 
environment; 

• the heightened focus on privatisation and the effectiveness and 
efficiency of service delivery; and an 

• increasing emphasis on accountability through public and open 
processes. 

A study undertaken across government departments and local authorities in 
Western Australia published in 1993 shows that government had already 
started to utilise benchmarking strategies in its search for Improved 
performance (Frost and Pringle, 1993). The study found that 30% of the 
organisations surveyed introduced benchmarking for improving productivity 
and/or efficiency; 19% for improvement in customer service and client 
satisfaction; and 15% to cost benefits and productivity. 

These public sector organisations tended to compare related government 
agencies more than any other source and related organisations in the private 
sector were compared more frequently than non-related corripanies. Some of 
the organisations chose overseas public sector organisations as their 
benchmarking partners. The study noted that "public sector organisations need 
to be more aware that true benchmarking Involves comparisons with industry 
leaders in any industry, be they related or non-related." (Frost and Pringle, 10, 
1993) 

Thirty six percent of the organisations used the benchmarking information to 
mod'fy existing practices and to change policies, 24% utilised the information 
as a guide for future management planning and decision making and 18% 
developed performance indicators and standards by reference to the 
benchmark data. Only 8% of the organisations positively identified the 
benchmarking process as being successful, although 34% indicated that it was 
moderately successful a further 32% rated its success between these two 
levels. Twenty six percent indicated that they considered that the exercise had 
been moderately unsuccessful 

Discussion 

As described in the forgoing, most benchmarking can be thought of as a 
quality assurance process that links targeted performance indicators for 
particular processes and aspects of organisational performance to a strategy 
for evaluating effectiveness and efficiency of processes and outcomes against 




12 



best practice in cognate organisations. The key development in benchmarking 
that takes it beyond the boundaries of traditional performance indicators is its 
incorporation of functional comparisons across organisations. In functional 
comparisons the focus is not on comparing the perfomiance of like processes 
or outcomes, but in asking whether novel and innovative developments in one 
organisation have a potential application in another organisation — the bar 
code example above. Benchmarking is also explicitly viewed by its 
practitioners as incorporating strategies for the further development of 
processes, that is it moves beyond the benchmark knowledge to the strategic 
planning and development required to achieve those standards. 



Pf rformance Indicators for System Performance^ 

The literature on performance indicators in education makes the distinction 
between those which are performance indicators, and those which are 
education indicators. The term performance indicator is often interpreted as 
referring to final outputs and the term education indicator is often used to refer 
to intermediate outputs and educational processes. Such a perspective is 
particularly prominent in the North American literature. However, there is good 
reason to treat all indicators in education as performance indicators. 

Indicator systems in education have been proposed in order to address a 
range of different issues. The main uses that have been s^jggested for them 
include: 

• assessing the impact of educational reforms 

• informing policy makers of the practices that are most effective for 
improving education 

• explaining causes of conditions and changes 

• informing decision making and management 

• stimulating and focussing effort 

• ensuring accountability 

• defining educational objectives 

• monitoring standards and trends 

• forecasting future changes. 

Each of the above uses of indicator systems in education is discussed in 
more detail in Cuttance (1989). Oakes (1986) has suggested that there are five 
types of information that indicators can provide at the operational level. These 
are: 

• performance information about the achievement of goals and 
objectives 



* This section is drawn from Cuttance {1994by^ J 
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• information on the features of the systerr that are most important in 
achieving particular goals and objectivas 

• policy relevant information 

• problem orientated information 

• information on central features of the system. 

Table 1 cross-classifies these five types of information against the nine 
purposes set out earlier for education indicator systems. The five types of 
information can be 8ub*divided into those which are derived from evaluations of 
the system and those which are derived from routine monitoring of the system. 
The category of 'policy relevant information' is applicable to information from 
both evaluation and monitoring sources. Also, a certain amount of 'problem 
oriented information' will be made available through the diagnostic and 
formative components of formal evaluation activities, but its main source will be 
from monitoring activities in the system. From table 1 it is clear that some of 
the purposes put forward for education indicators draw more heavily on 
mon,toring activities and others draw more heavily on evaluation activities. 
Thus, an indicator system which encompasses all nine purposes would need to 
gather information from both formal and informal evaluations and from routine 
monitoring of the system. 
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Table 1 Purposes of education indicator systems and the types of 
information they need to provide 
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X 


X 


X 
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strategic Information to Assist the Improvement of School Systems 

The concept of benchmarking is normally applied to Individual organisations, 
units within organisations or individual processes. In the context of school 
systems schools and individual processes within schools may be the focus of 
benchmarking, but the systems themselves could also be the subject of a 
benchmarking exercise. 

Cooley (1983) discusses the information requirements for improving 
education systems. The research literature on the initiation and management 
of change and decision making process in large systems provides a framework 
for determining the key features of information required as part of the 
improvement process. The literature on the utilisation of research in the policy 
making process and models for educational evaluation can be viewed for the 
present purposes as particular strands within these perspectives. In the 
context of organisational change, the work of Berman and McLauchlin (1974) 
and Elmore (1983) provide models that bridge these two perspectives. Fullan 
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(1991) provides a detailed overview of the factors that are of primary Interest in 
successful organisational change in complex educational organisations. 

The 'rational' model of decision making posits that systems make decisions, 
and act on them, in response to data and frameworks for deliberative action. 
Action-based models, such as those embodied in change strategies involving a 
significant degree of stakeholder participation posit that the 'political' 
environment of the organisation is a determining factor in both shaping the data 
and the orientation to action. 

The quality management literature emphasises gradual or continuous 
improvement rather than structural or large scale change. Quality management 
approaches can be criticised for a failure to recognise the necessity for both 
continuous improvement and significant structural change in complex 
organisations (Cuttance, 1993a; Nohria and Berkley, 199>!). Both continuous 
improvement and structural change require ongoing monitoring systems to 
assess progress towards desired outcomes. 

Cooley (1983) provides a discussion of the key factors for successful 
monitoring in school systems. He argues that there are two primary features of 
such systems for the continuous monitoring of progress: 

• a client orientation, and 

• a systems approach to program improvement. 

Client Orisntatlon ^ 

A client orientation is necessary if the data made available from the 
monitoring system are to influence the alternative perspectives that decision 
makers bring to the situation. Without a clear and agreed position of who the 
clients are, and there may be more than one type of client, the information 
gathered through the monitoring system will have little relevance to decisions 
that focus on improving the outcomes of the system. The value of information 
in relation to outcomes for clients is not that the data determine priorities or 
settle policy issues but that they permit those issues to be argued more 
productively (Cooley, 1983). An effective client orientation in the monitoring 
system requires that the system itself allow for interaction with clients in the 
data gathering process. 

The effective use of new information from monitoring activities requires 
strategies for ensuring that decision makers digest the data available. The data 
must be integrated into the decision maker's working knowledge of the situation 
(Kennedy, 1982). Decision makers at different levels in an organisation have 
different sets of 'working knowledge' and it is therefore critical to the utilisation 
of monitoring information that its presentation be carefully tailored to these 
different sets of knowledge. Few decision makers at any level from the 
classroom to the chief executive in a school system have the luxury or the time 
to read long and detailed reports. Further, decision makers at the different 



16 



levels requires information about different aspects v ) the performance of the 
system. 

The information required for affective decision making and action need not 
aspire to be generalisable to other situations and contexts. Indeed, its utility is 
significantly dependent on its contextual nature. "What seems generalisable 
are the approaches ... for generating information needed to [respond] to 
clients in a particular context at a particular time." (Cooley, 1983; 7) 

A Systems Approseh to Improvsmsnt 

The systems approach that Cooley describes is focussed on the continuous 
use of indicators to monitor performance and adapt practice to the 
requirements of the situation. This is contrasted with the alternative of 
employing periodic summative evaluations to assess the success of discrete 
programs. The approach of summative program evaluation provides a static 
view of the performance of a program. Programs, however, are dynamic and 
are continually impacted by other extant programs, new programs and general 
turbulence and influences in the system's environment Further, program 
evaluations take a significant time to complete and it is not uncommon to hear 
that decision makers had to make the decision before the information from the 
evaluation was available. 

This does not mean that there is not a role for summative program 
evaluations in assessing the performance of school systems, rather that the 
role should not be seen as one which can provide responsive information of a 
dynamic nature as required for management decision making. Summative 
program evaluations aim to provide infrequent but accurate information, 
whereas the needs of decision makes is for frequent information, even if it is 
somewhat less accurate. The power of the data provided from a monitoring 
system is gained from its contribution of contextual understanding to the 
working knowledge of the decision maker and the corroborative information it 
contributes to indicate situations where performance has moved out of the 
expected range in specific sectors of the system, or over time. Such monitoring 
information allows the decision maker to take action as appropriate in response 
to information that things are not proceeding according to the way that they 
were expected to do so. The accumulation of information from such monitoring 
systems can provide the basis for establishing the areas in which a system 
needs to focus its attention in improving overall performance. 

Monitoring systems can also provide an indication of the distribution of 
performance throughout a school system in relation to a particular indicator. 
This supports meta-level inferences indicating whether the system is 
performing uniformly or whcVier there are performance issues that require 
attention in particular parts ot '.he system. For example, In a school system 
there may be particular performance issues in the high school sector that are 
not found in the primary school sector. The adaptlveness of teaching strategies 
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to student learning styles is one such issue of relevance to Australian school 
systems. 

Where monitoring over time provides information that performance is 
deteriorating or not meeting expectations after focussed corrective action has 
been taken there is a need to establish a broader understanding as to the 
nature of the problem. This provides an important and fundamental role for 
program evaluation. The contribution of program evaluations to system 
performance rests less on summative information than on formative and 
diagnostic analysis of the issues responsible for performance. Clearly, an 
assessment of the summative performance of a program is required before any 
analysis of the issues impeding the achievement of program objectives can be 
undertaken, but this by itself is of little utility in managing the perfomnance of a 
school system. Decision makers require an analysis of the potential responses 
they could make to particular performance problems. The solution should not 
be bound entirely by the program under evaluation, as management also 
requires to assess the likely impact of any response on other programs. 

The outcome of an effective performance indicator system, therefore, is 
guidance for priority setting and the provision of data to inform appropriate 
action, which may be focussed corrective action. Over time a performance 
indicator system should alio provide information about how the school system 
is responding to changes in its external environment. This information should 
be extrapolated to provide limited 'over the horizon radar'. 

Discussion 

The role of performance indicators in the context of benchmarking is 
indicated by the fact that all the purposes put forward for performance 
indicators can be met through benchmarking, although this is not to argue that 
any organisation should seek to do so. The role of benchmarking and the 
purposes of performance indicators can be considered to be congruent if the 
content of the information provided is appropriate. 

The information provided by the best practice statements in school reviews 
is reflected in a series of indicators referred to as 'pointers', an example of 
which is provided in table 2. The difference between these indicators and 
traditional performance indicators lies in the interpretation usually made of such 
data. The best practice pointers are most definitely not a comprehensive 
description of the characteristics or qualities of the activities being recorded. In 
reality this is also true of traditional performance indicators, although this is 
often forgotten when it comes to assessing their implications for policy and 
practice 
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Table 2 Example of Pointers for a Best Practice Statement 



Dimension of practice 


Teaching & Learning 


Aspect of practice 


Planning & Implementation 


Best practice statement 


Teaching programs provide for 
appropriate sequencing and 
continuity of learning for each student 


Pointers 


• A clear set of markers for 
s&quential skill development 

• Recording of student progress 
relative to previous learning 

• Recognition of progress along each 
student's learning curve 

• InHiuiHiial loamlna Droflles for each 

IIIUIVIUUGII lOOl 1 III ISJ k/IWIIIWW IWI WMWI 1 

student 

• 'vidence of a progression of 
SKills/knowledge/concepts in the 
teaching program 

• Ongoing assessment of each 
student's progresst 



Program evaluations conducted as an adjunct to a performance indicator 
and monitoring process provides much of the information that benchmarking 
requires to carry it forward from the analysis stage outlined earlier to the 
understanding of why a process or organisation is performing as it is. 
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Benchniarking a School System 

There have been only two published articles in education that refer to 
benchmarking of the type described in this paper (Doyle, 1993: Faidley and 
Musser, 1991). The essential elements of benchmarking as described above 
that will be discussed in this section deal with both Internal and external forms 
of benchmarking in the NSW Department of School Education 

The benchmarking process is an integral part of a broader quality assurance 
system, The quality assurance system provides a framework for reviewing the 
performance of all schools on a regular basis and for determining the focus for 
further development in each individual school. Quality assurance school 
reviews are undertaken as a collaborative exercise between the school and its 
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community with assistance from the Quality Assurance Directorate. School 
reviews focus on the performance of the individual school, factors impeding or 
enabling that performance, and the further development required to better to 
meet the educational needs of the local school community (Carbines, 1934; 
Cuttance, 1993b). 

An essential element in the quality assurance system is the establishment of 
a set of statements describing best practice throughout the school system 
(Highett, 1994). The best practice statements have been developed 
collaboratively between practitioners and researchers in the NSW school 
system. Research on effective practices in other system and on the variation in 
practice in this school system has been used to augment practitioner's 
understandings of the most effective practices in different contexts. This 
external research focus allows the benchmarking system to meet the rrajor 
requirements of external generic benchmarking as described earlier in this 
paper. The use of practitioner understandings of best practice in the school 
system as a means of reviewing the performance of individual schools means 
that the process has significant elements of internal benchmarking as 
described earlier in this paper. 

The benchmarking process focuses on the value-added outcomes for 
individual schools as a way of determining the stage of development in terms of 
a performance development cycle (Cuttance, 1994a). This is crucial given the 
findings of the American Quality Foundation study on best practice that there 
are few universally effective quality management practices and strategies, and 
that particular strategies and practices have greater marginal effectiveness at 
different stages of the performance development cycle.. 

The best practice statements have been developed across three key 
dimensions of school practice: teaching and learning, management and 
governance, and leadership and culture. An important element of the best 
practice framework is the evaluation of the effectiveness of practices at 
different stages of the performance development cycle. This will be undertaken 
through a research analysis of the effectiveness of individual best practices in 
relation to the value-added performance of individual schools. In essence, this 
takes the mainstream research methodology of school effectiveness a step 
further than that which has normally been the case. Many school effectiveness 
studies have researched the effectiveness of individual practices against 
performance in terms of student outcomes, but none have considered this in 
the context of performance development cycles for schools. 



The Best Practice Statements 

The best practice statements have been collaboratively developed by 
practitioners and researchers in the school system. They are designed to 
reflect the outcomes of effective practices in schools in different cont^^xts. Their 
primary purpose is to provide an agreed framework for the negotiation of focus 
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areas for the indepth review of the performance and development of individual 
schools. These reviews are undertaken collaboratively with achools and their 
school communities and the findinga published in public reports. The 
statements take into account the fact that schools work In different contexts and 
move through atages of performance development. The Interpretation of the 
best practice statements is context sensKlvf to that they relate to the particular 
client population of the tchool. They provide a background against which the 
school community can test their strategic plan and determine areas for further 
development in the school. 

The statements themselves are neither prescriptive nor comprehensive 
descriptions of the way to implement best practices. Rather, they describe the 
key characteristics that are expected to be observable as the outcomes of 
effective practices in terms of student learning. Thus, they respect the 
professional integrity of staff in achools to make their own judgements about 
the way in which the statements apply to their context. In terms of the literature 
on performance indicators these statements are high-inference indicators of the 
outcomes of the practice and functioning in schools across the system. 

In drawing up the framework for the best practice statements the project 
teams asked the following three questions: 

• What are the major dimensions a school's operation? 

• Within these dimensions, what are the major areas operation and 
functioning that define the scope of that dimension? 

• How would teachers and others in schools describe^best practices in 
that dimension and what 'pointers' would indicate that best practices 
were a part of the paiticular school's operation? 

During the development process the best practice statements were trialed in 
a large number of schools. To date about three thousand school based staff 
and others have provided feedback on the draft statements. Each of the 
dimensions of the best practice statements are set within an overarching 
context of the mission for the school system. 

The Benchmarking Process 

The following analysis is set out in the framework for the stages of a 
benchmarking process as attributed earlier to Camp (1S89). The first stage oi 
the process is planning. The design of the best practice framework is intended 
to provide the basis for schools to identify what they want to benchmark. It is 
not intended that individual schools will wish to benchmark against all of the 
best practice statements. The framework provides a synthesis across 
individual schools within the overall school system as the referent for the 
benchmarking analysis. The data collection process will capture the 
professional evaluations of school-based staff of the performance of their own 
school against the best practice statements. 
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Stage two of the benchmarking process is an analysis of the information on 
the performance of the organisation being benchmarked against the referent 
organisation. This analysis will involve both an evaluation of the effectiveness 
of practices and of student outcomes in temis of the performance of the 
individual school. Full details of the development of the best practice 
statements and the mode of their use are available in Highett (1994) and 
Carroll (1994). Dawson (1994) describes their use in improving the 
performance of individual schools. 

One of the key uses of the best practice statements in the benchmarking 
process is to determine focus areas based on the processes and functions for 
which the school wants to improve its performance. Following this the review 
undertakes an Indepth evaluation of the focus areas to provide information 
about the context and the factors responsible for impeding current 
performance. This information is then linked to that available across schools 
about how the best practices of schools lead to improved performance in such 
areas. 

The information made available to the review is discussed by the review 
team in structured debriefings throughout the period of the review, and 
communicated to the broader school community in a preliminary report at the 
end of the site visit pnd a formal public report provided soon after the 
completion of the visit. The review process develops a set of recommendations 
(or the further development and enhancement of the performance of the school. 
These recommendations are a feature of the published fintil report from the 
review. 

The principal of the school is accountable for implementing appropnate 
action plans to follow through on the findings from the review process. Where 
appropriate the principal will call on external support available through the 
programs and se' Aces provided by regions to schools. Directors of schools 
ara responsible for ensuring the provision of appropriate regional support for 
schools to improve their performance. Each school develops an action plan 
and incorporates this within its general planning and management processes. 
A key feature of such action plans is a monitoring process to assess the 
effectiveness of the implementation designed to improve the school's 
performance. 

The research that is to be undertaken to assess the effectiveness of 
practices in schools at different stages of the performance development cycle 
will be used in recalibrating the benchmarks. Validated information on best 
practices in terms of their relevance to the improvement of student learning 
outcomes will be provided from this research. 

Once fully implemented the quality assurance system, including the 
benchmarking process, will be part of the normal cycle of review-development- 
monitoring-review in all schools. 
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At a system level the benchmarking of Individual schools provides 
system wide information about the attainment of best practice. The gaps 
between current practices and best practices at system level provide the basis 
for enhanced decision making to ensure that resource alk>catk>ns, partteularly 
in the area of training and development of staff, target those areas In whteh 
practice Is furthest from current best practtee. Figure 1* Indicates the type of 
information that is provkled at system level. Practices at the top of the figure 
are those for which the smallest gap exists between current practice and the 
best practice statements across schools in the system. As performance 
Indicators these benchmarks are different from the traditional performance 
Indicator in school systems because they have the characteristic of assessing 
performance in relation to best practice. 



Discussion 

The American Quality Foundation study assessed the efficacy of 
benchmarking practices as part of Its overall quality management study. The 
study found that the greatest benefits of benchmarking were for organisations 
that already had high levels of performance. However, the study also 
suggested that organisations at other stages of their performance development 
cycle may benefit from benchmarking M they focus on benchmarking against 
organisations at a similar stage of their performance development cycle, or 
marginally ahead of them, rather than the best-ln-class orgdnisations. This is 
an important caveat to receive the wisdom on the effectiveness of 
benchmarking and may explain why some of the public sector organisations 
that have already tried benchmarking have not found it to be completely 
successful as a strategy for improving their performance. 

Benchmarking in the public sector has been in use for some time, althougb 
it is not yet widespread. Careful consideration needs to be given to ariy 
benchmarking exercise to determine clearly the objectives and the context in 
which it is to be applied. In particular, the outcomes expected from the 
exercise need to be clearly understood. Where these include 8 focus on the 
acquisition of innovative strategies that could be applicable to the organisation, 
an external partner snould be chosen for functional benchmarking purposes. 

Benchmarking can be linked to the practice of using performance indicators 
that is widespread in the public sector. However, it requires two significant 
developments beyond the traditional use of performance indicators. The 
performance indicators themselves must encompass both performance in 
terms of outcomes and the effectiveness and efficiency of processes 



5 The data on which this figure is based are fictitious 



23 



responsible for producing those outcomes. Few performance indicators in the 
public sector address this latter aspect of the effectiveness and efficiency of 
processes. The interpretation of the performance indicator information must 
also take best practice as its referent standard. Thus, the performance 
indicators should be utilised as a basis for assessing the gap between current 
practice and best practice. P^jrformance indicators that are an integral part of a 
benchmarking system address each of the key purposes of most public sector 
performance indicator systems. 

Benchmarking in the NSW Department of School Education is one element 
of an integrated quality assurance system. The benchmarking process is used 
in the context of indepth reviews which focus on the gap betw een current 
practice and best practice in schools. The benchmarking information is used to 
identify key focus areas for the reviews. Extensive evaluation is then 
undertaken in these key focus areas to understand the factors responsible for 
the schools current performance and to clarify the strategies that it might 
pursue in its improvement of performance and for further development to meet 
community needs for education. 

The outcomes of this benchmarking process also provide vital information at 
the system level to support decision making at>out the allocation of resources. 
It also provides information that supports the program evaluation function in 
targeting gaps in the provision of programs and services to schools to meet 
government policy for schooling. 
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